In a wide variety of image sequence processing and analysis tasks, there is a great need for an accurate method for tracking the intensity and motion of a portion of an image throughout an image sequence. This portion, called the reference region hereinafter, may correspond to a particular object or a portion of an object in the scene.
Tracking the boundary of an object has been discussed in M. Kass, A. Witkin, and D. Terzopoulos, "Snakes: Active Contour Models", International Journal of Computer Vision, volume 1, no. 4, pp. 321-331, 1988; F. Leymarie and M. Levine, "Tracking Deformable Objects in The Plane Using An Active Contour Model", IEEE Transactions Pattern Analysis and Machine Intelligence, volume 15, pp. 617-634, June 1993; K. Fujimura, N. Yokoya, and K. Yamamoto, "Motion Tracking of Deformable Objects By Active Contour Models Using Multiscale Dynamic Programming", Journal of Visual Communication and Image Representation, vol. 4, pp. 382-391, December 1993; B. Bascle, et al., "Tracking Complex Primitives in An Image Sequence", in IEEE International Conference Pattern Recognition, pp. 426-431, October 1994, Israel; F. G. Meyer and P. Bouthemy, "Region-Based Tracking Using Affine Motion Models in Long Image Sequences", CVGIP: Image Understanding, volume 60, pp. 119-140, September 1994, all of which are herein incorporated by reference. The methods disclosed therein, however, do not address the tracking of the local deformations within the boundary of the object.
Methods for tracking local deformations of an entire frame using a 2-D mesh structure are disclosed in J. Niewglowski, T. Campbell, and P. Haavisto, "A Novel Video Coding Scheme Based on Temporal Prediction Using Digital Image Warping", IEEE Transactions Consumer Electronics, volume 39, pp. 141-150, August 1993; Y. Nakaya and H. Harashima, "Motion Compensation Based on Spatial Transformations", IEEE Transaction Circuits and System Video Technology, volume 4, pp. 339-357, June 1994; M. Dudon, O. Avaro, and G. Eud; "Object-Oriented Motion Estimation", in Picture Coding Symposium, pp. 284-287, September 1994, CA; C.-L. Huang and C.-Y. Hsu, "A New Motion Compensation Method for Image Sequence Coding Using Hierarchical Grid Interpolation", IEEE Transactions Circuits and System Video Technology, volume 4, pp. 42-52, February 1994, all of which are herein incorporated by reference. However, these methods always include the whole frame as the object of interest. They do not address the problem of tracking an individual object boundary within the frame.
U.S. Pat. No. 5,280,530, which is herein incorporated by reference, discusses a method for tracking an object within a frame. This method employs a single spatial transformation (in this case affine transformation) to represent the motion of an object. It forms a template of the object, divides the template into sub-templates, and estimates the individual displacement of each sub-template. The parameters of the affine transformation are found from the displacement information of the sub-templates. Although this method employs local displacement information, it does so only to find a global affine transformation for representing the motion of the entire object. Therefore, while it tracks the global motion of an entire object, it cannot track any deformations that occur within the object (i.e., local deformations).
Although the presently known and utilized methods are satisfactory, they are not without drawbacks. In addition to the above-described drawbacks, they also do not take into account the effects of frame-to-frame illumination changes.
Consequently, a need exists for an improved tracking technique that can track objects within a scene which are undergoing local deformations and illumination changes.